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1. 



DECLARATION OF DR. LUIS SERRA UNDER 37 C.F.R. 1.131 

I am Luis Serra, Ph.D., the inventor of the above-identified patent application. 



2. I make this declaration to establish the date of my invention as being prior to May 18, 
2001, the filing date of U.S. Patent No. 6,826,297 to Saito et al ("Saito"), which has been cited 
against the above-identified application as prior art. 

3. Attached hereto as Exhibit A is a true and correct copy of an original Disclosure of 
Invention ("the Disclosure of Invention") for the invention described in the above-identified 
application. I prepared this for submission to our original patent attorneys. The Disclosure of 
Invention was written by me (and an assistant who is not an inventor) on March 8, 2001, and 
comprises a title page and three pages of disclosure. The Disclosure of Invention was written in 
Singapore, my country of residence, a WTO member country. 

4. Attached hereto as Exhibit B is a true and correct screen capture of a display on my 
computer, which shows the results of my having clicked on the "Properties" button associated 
with this document, overlayed on a background of the first page of the document itself which I 
had open on my computer screen at the time I captured the screen. Exhibit B shows that the 
document of Exhibit A was created on March 8, 2001 and modified later that same day. 

5. The Disclosure of Invention clearly describes volumetric data sets being compared in 3D 
and rendered fused, that is, displayed simultaneously on the same virtual space. Since the 
volumes are fused they can be compared against each other by a combination of rendering 
techniques. 
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6. The Disclosure of Invention further describes displaying two 3D data sets together, such 
as different modalities of the same object, using a variety of possible views for each modality. 
These include, for example, using the cutting planes of one volume to reveal the other, such as is 
shown for example in Figs. 1 1-28 of the above-identified patent application, displaying a cutting 
plane from each volume, displaying one semi-transparently, or using mixed visualization modes 
for each object. 

7. The Disclosure of Invention further describes several display modes for the invention, 
including: roaming mode, cropping mode, same planes locked together, one slightly above the 
other and slightly transparent, and a checkerboard comparison mode (such as is shown in Figs. 
3a and 3b of the above-identified patent application). 

8. Given the date on which I wrote the Disclosure of Invention, which clearly describes the 
invention which is the subject of the above-identified patent application, I respectfully request 
that the Saito reference be removed as a prior art reference against the above-identified patent 
application under 37 C.F.R. 1.131. 



I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code and that such willful false statements may jeopardize the validity of the above-identified 
application or any patent issued thereon. 



Dated: May 30, 2006 
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Disclosure of Invention 



Title: Method to verify the accuracy of the registration and 
segmentation of 3D volumetric images using visual inspection and 

3D manipulations. 

Authors: Luis Serra, Ng Hern 
3/8/01 
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Title: Method to verify the accuracy of the registration and 
segmentation of 3D volumetric images using visual inspection and 

3D manipulations. 

Abstract 

Method to verify the accuracy of the registration and segmentation of 3D volumetric 
images using visual inspection and 3D manipulations. 

Background of the Invention 
Field of Invention 

This invention relates to the field of computer graphics and human-computer user 
interface systems. More particularly, this invention relates to a 3D interactive tool to 
verify visually the accuracy of volumetric operations like registration of multimodal 
3D volumes and the segmentation of sub-volumes. 

2. Art Background 

Describe the problem: Comparing 3D objects. 

Operations on volumetric data usually involve "verifying" or "comparing" two or 
more volumes, to measure for example how accurate a registration between two 
volumes is. 

Another important operation in volumes is "segmentation", which relates to the 
process of subdividing a volume into smaller segments like a tumor from a brain, be it 
for the purpose of quantifying its properties, or to study its shape or fit within the 
larger volume. Whatever the method used to obtain a segment out of a volume, there 
is the need to determine how accurate a segmentation is with respect to the original 
volume, and this requires in the end human intervention. 

Why needed? 

Art approach is to break the problem down to comparing in 2D, one cut of the object 
(a slice) against another, or a cross section of a 3D polygonal mesh against another 
cross section, or a volume slice. This gives only a partial view of the entire problem, 
and requires tedious, plane by plane comparison. 

We propose to compare volumes directly in 3D space, using a combination of 
visualization techniques and 3D interaction techniques as described in this patent. 

Background 

Volumetric data is obtained from digitizing or scanning objects in the real world, like 
for example a Computer Tomography (CT) scan of a person's head. The scans result 
in a stack of slices. The slices can be displayed as 3D reconstructions by means of a 
process called volume rendering. For complex objects (e.g., a human head) it is 
becoming increasingly common to require more than one scan modality since 
modality can only reveal a partial aspect of the whole. For example, a CT scan is good 
to reveal the bone structure but inadequate to show well the soft tissue (for example, a 
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tumor). To view the entire object, it is then important to first, register together the 
different modalities of the same object, and second display them together (fuse). 
Registration requires accurate relative positioning of the two volumes. Fusion requires 
visual correctness of the resulting image. 

Definitions 

Visualization modes: 

Triplanar 

Full 

Cutbox 

Monoplanar 

Summary of the Invention 

A method is disclosed that allows a user to compare two 3D volumetric objects 
against each other. 

The method uses the 3D interaction environment of the Dextroscope [define]. 

The method is useful to determine the accuracy of segmentation and registration 
between to volumetric data sets. 

In this method, volumetric data sets are compared in 3D, with a stereoscopic display, 
and rendered "fused", that is, displayed simultaneously on the same virtual space. 
Since the volumes are fused, the can be compared against each other by a 
combination of rendering techniques under the control of 3D interaction tools. The 
volumes reside in the same virtual space. 

Messy from here onwards. . . 

The user controls the cutting planes of one volume to reveal the other, or move two 
cutting planes simultaneously (one for each volume). The cutting planes reveal a 
cross-section of one volume that can then be compared against another cross-section 
of another volume. 

If the object closer to the user's viewpoint is opaque, it obscures the object 
underneath, and the comparison cannot be achieved. In this case, the validation tool 
uses different approaches: 

• transparency control of the object. Either object (but usually the object closer to 
the viewpoint) can be turned semi-transparent under user control so that the two 
objects can be seen simultaneously and thus compared. 

• Mixed visualization modes for each object. Either object can be visualized in a 
mode that allows seeing partially through the object. For example, one object can 
be displayed in triplanar mode while the other is displayed in full mode. This case 
applies when one is comparing for example a head CT to MR registration, and the 
user wants to see how good the fit of the skull bone is. The CT can be displayed 
full, and the MR in triplanar. The CT only displays the bone (not the surrounding 
tissue) and the MR covers the whole head, so that the outer layers of both sets are 
visible. Or one object can be in cutbox mode while the other is in full mode or 
triplanar mode. 
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The validation tools also works in "volume to polygon" mode, to verify the accuracy 
of segmentations (contours extracted from volumetric images). 

The validation tool also work in "volume to real object" mode, to verify the matching 
of computer-generated images against real objects seen through a half-silvered mirror 
or a video camera feed. 

Independent of the visualization mode of each data, the method has several modes of 
operation 

• Roaming mode. The two objects are cropped along the same planes 

• Cropping mode. Each object is cropped along different planes. 

• Same planes locked together. 

• One slightly above the other & slightly transparent (the one above) 

• Checkerboard comparison mode. The two objects are displayed alternatively to 
allow a general view of the fit. 

Extensions: 

• More than two objects 

• Time variant comparisons. Dynamic against static objects. 



